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This paper presents both the pedagogical 
design principles and the challenges faced dur¬ 
ing the development of a mobile app, Chinese 
Characters First Steps, to support recognising, 
learning and practising writing Chinese char¬ 
acters at beginners' level, including the diffi¬ 
cult balance between pedagogy and technical 
affordances. We also present the results of a 
research study which collected quantitative 
data from over ljo users of the app as well 
as qualitative data from interviews with four 
users. The results offer a profile of language 
learning app users and very positive attitudes 
towards learning with apps. They also reveal 
users' reasons for learning Chinese, patterns 
of learning, expectations of the app, and evalu¬ 
ation of the different features for their own 
personal learning. Finally, the paper concludes 
that the chosen design principles for the app 
were appropriate for its purpose. 
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1. Introduction: the app revolution 

When the first generation iPhone was intro¬ 
duced in 2007, few could have predicted 
the impact it would have on education. As 
Godwin-Jones (2011) points out, the iPhone 
was a game-changer not only in terms of its 
functionalities, but because other competi¬ 
tors followed and created similar devices. 
Since then, and with the addition of the 
iPad - introduced in 2010 - and similar 
devices, there has been a proliferation in the 
popularity and ownership of smartphones 
and tablets that can carry out a large vari¬ 
ety of educational activities within a single 
device. Although the devices are clearly 19 
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necessary for this to happen, the other pillar of this revolution in mobile learning is the 
availability of the apps developed to be used with them. 

As soon as these apps were released, educators began to see the potential for teaching 
using such devices, including ubiquitous learning opportunities, ease of use, collaboration, 
content generation, and productivity enhancement (Murphy, 2011). But does this potential 
become reality? And what are the implications for language learning in general and for 
more complex languages such as Chinese? 

2. Literature review 

This section will present the challenges of learning Chinese, current theory on the use of 
mobile apps for language learning, and the evidence found so far in research studies about 
their efficacy for teaching and learning Chinese. 

2.1 Challenges of learning Chinese 

It is well established in the research of teaching Chinese as a foreign language (CFL) that, 
due to its unique features, it presents the following key challenges to language teachers and 
learners: i) tones; ii) character recognition and writing; iii) lack of obvious correspondence 
between the character script and the sound (Hu, 2010; Liang & van Heuven, 2007; Wang & 
Kirkpatrick, 2012). Tones and characters are two of the biggest hurdles at beginners’ level 
(Liang & van Heuven, 2007; Jongman et al, 2006; Hu, 2010; Xiao, 2010). Not only are there 
too many similar sounding words in Chinese (Lii, C, 2010), but also many words sound 
exactly the same (with the same syllables and same tones) but mean different things and 
have different character representations. Learning characters is crucial in meaning identi¬ 
fication. However, characters are perceived as difficult because of their sheer number (Hu, 
2010) and complexity of strokes (on average a character consists of about 12 strokes). Many 
beginners, face-to-face or learning at distance, feel that they are learning two systems at 
the same time - pinyin (phonetic transcription of the characters) and characters. In this 
process, they need to do four mappings (see Table 1 with an example for the word for west ): 

Although there has been extensive research looking into character learning strategies 
(Ke, r998; Lu, 2009; Yin, 2003) as well as issues concerning tone production and recogni¬ 
tion (Liang &van Heuven, 2007; Jongman et al, 2006; Hu, 2010), not many researches have 
looked into the effective use of new technologies to support and engage learners in doing 
the four mappings as illustrated in Table 1. 

2.2 Apps for language learning 

2.2.1 Potential and challenges. Many authors have highlighted the potential of smartphone 
and tablet devices as well as mobile apps for language learning, based on the theoreti¬ 
cal principles and evidence from the field of Mobile-Assisted Language Learning (MALL) 
(Burston, 2014; Godwin-Jones, 2011; Golonka et al, 2012; Kim & Kwon, 2012; Kim, 2013; 
Lafford, 2011; Lys, 2013; Shinagawa, 2012; Sweeney & Moore, 2012; Yang & Xie, 2013). This 
potential derives from features such as larger screen size, responsive touch screen, enhanced 
text-entry, high-quality audio and video playback, picture, video and audio recording and 
editing, voice recognition, enlarged storage, faster connectivity, GPS, accelerometers and 
20 Bluetooth connections (Godwin-Jones, 2011), all within one device. 
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Table 1: An example illustrating the four mappings learners need to do to match pinyin 
spelling, pronunciation, character script and translation. 



Pinyin 


Character 


Mapping 

spelling 

Pronunciation 

script 

English 

0) 

xl 

Y is pronounced a bit like 'ch' in the 



Pinyin spelling with 


English word 'machine'; 'i' is pronounced 



pronunciation 


like 'ea' in 'tea'. 



(2) 

Pinyin spelling with 
character script 

xl 


a 


(3) 

xl 

'x' is pronounced a bit like 'ch' in the 

a 


Pinyin, character 


English word 'machine'; 'i' is pronounced 



with pronunciation 


like 'ea' in 'tea'. 



(4) 

xl 

Y is pronounced a bit like 'ch' in the 

a 

west 

The above three 


English word 'machine'; 'i' is pronounced 



with English 


like 'ea' in 'tea'. 




Criticisms of the design of language learning apps include too many translations, lack 
of sound or pictures, poor navigation and user-interface design, and little use of the unique 
properties of smartphones such as the motion sensor, location detector, and, in particular, 
connectivity with other users (Godwin-Jones, 2011; Sweeney & Moore, 2012; Burston, 2014). 
Burston (2014) found that language learning activities on mobile apps have mostly repli¬ 
cated what had been done before with other technologies and limited to basic flashcards, 
multiple choice, blank filling, drag and drop and vocabulary and grammar drills and games. 

He concluded that “pedagogically, nothing new has been done with smartphones that has 
not already been done with earlier mobile devices" (Burston, 2014, p. 108). In their review 
of language learning mobile apps, Kim & Kwon (2012) highlighted that most apps focus on 
cognitive processes (recognition, recall and comprehension) and receptive language skills 
rather than socio-cognitive activities, with little collaborative learning; they also found the 
activities not as well developed as they could be and mostly teacher-directed, with focus 
largely on vocabulary. 

2.2.2 Evidence of app use for Chinese language learning. Although some educators and 
students are quick to adopt new technologies into their teaching and learning, it is neces¬ 
sary to evaluate the real potential of the tools. 

As Golonka et al (2014) correctly point out: "in spite of an abundance of publications 
available on the topic of technology use in FL learning and teaching, evidence of efficacy 
is limited" (Golonka et al, 2014, p. 70). Studies on the use of mobile apps for Chinese lan¬ 
guage learning are scarce. Shinagawa (2012) described some of the benefits that mobile 
apps can bring to learning non-western characters. These include the iPhone's own Chinese 
handwriting recognition function, dictionaries, and quizzes. Among the attempts to find 
evidence of Chinese learning with apps, Wong et al (2010) carried out a research study in 
Singapore, providing their students with smartphones to photoblog Chinese idioms and 
subsequently base collaborative activities around them. They encountered varied levels of 
participation (partly due to parental negativity towards the use of smartphones by school 21 
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children) as well as technical problems. Despite this, their students reported more engage¬ 
ment with Chinese idioms - although post-study performance was not tested. Yang and Xie 
(2013) also researched learning Chinese idioms, using iPads with heritage learners studying 
at a university in the USA. The participants had very positive impressions of the learning 
experience. A post-test after the activity showed that almost all students could recall the 
idioms practised, however, after two weeks, the recall had fallen to 40%. 

As discussed in section 2.1, character recognition and writing present a challenge for 
learners of Chinese as a foreign language. Technology can facilitate their learning or pro¬ 
vide methods to help overcome the need to handwrite them (Lai & Gu, 2011), but very few 
studies have looked into whether and how apps can support character recognition and 
writing. Huo (2013) found that stroke order is still relevant when learning Chinese with 
the use of technology, and claimed that stroke order helped memorise characters. Based 
on the research she carried out with learners of Chinese at a US university, Chung (2013) 
claimed that practising Chinese characters with an iPad had led to the students learning 
basic stroke order, familiarising themselves with structural components of the characters, 
and gaining knowledge of the history and culture behind the words. Her students liked 
the fun, game-like features of using the iPad and this encouraged them to continue with 
the activities. Some of her students commented on the ease and convenience of using a 
mobile device for learning, for example whilst commuting, allowing them to make the 
most of their time outside the classroom. Chung concluded that using the iPad facilitated 
the integration of learning Chinese into the students' daily lives. 

3. Design principles and development of the app 

Beginners’ Chinese was introduced to the curriculum at The Open University (OU), a UK- 
based distance learning university, in November 2009. As distance learners, our students 
face additional challenges to those mentioned in 2.1, such as lack of physical and visual 
presence of tutor and fellow students (Hurd, 2005); lack of immediate personal feedback, 
and limited face-to-face speaking and interaction opportunities (Kan & McCormick, 2014; 
Stickler & Shi, 2013). Due to these challenges, the beginners’ Chinese course team searched 
for innovative tools to support their students in meeting the challenges of learning tones 
and characters. However, in 2011, there was limited number of apps designed for Chinese 
character learning. Amongst those in the market at the time, most of them had less than 50 
randomly selected characters. The team was not able to identify an app that was designed 
to learn characters in a progressive and personalised manner. To address this and other 
shortcomings of the apps at the time, such as over-emphasising the fun element (showing 
the rain drops with the character for rain) whilst neglecting integrating different aspects of 
learning characters: pronouncing, listening, writing and recognising characters, the team 
worked with the app developers at our university to create a new app 'Chinese Characters 
First Steps'. Although the app content is of use to students taking the OU course, its use 
is not compulsory. This is because the university does not assume that students will have 
access to (or own) a smartphone or tablet that can run the app. This section presents the 
principles behind the app design and the stages of production that the app went through. 


22 
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3.1 Design principles 

Sweeney & Moore (2012) stated that the critical success factors in the design of apps for 
language learning are pedagogy and technology. The principles behind our app design 
included: 

rjc Bite-size learning and mobility 

It was agreed that small learning bites would work best due to small screen size and 
devices often being used on the move, as they can also maximise the number of learning 
situations in which the resource can be used (Kovalchick & Dawson, 2003). In addition, 
the mobile learning experience has been described as “highly fragmented" (Trifonova 
& Ronchetti, 2003) since its early days, and best design should take this into account 
by providing activities that can be completed in a short amount of time to make use of 
‘dead time'. Hence, each listening and learning activity in the app consists of only ten 
matching guestions. 

rtc Progressive learning 

There are altogether 20 lessons with 400+ characters in the app, and each lesson consists 
of an average 20 characters. All the characters are covered in the OU beginners’ Chinese 
course, which are the most commonly used characters at beginners' level. The 20 les¬ 
sons build upon each other in the same chronological order as in our course materials 
so the characters learnt in previous lessons are used to build new words in later lessons. 

rtc Integrating writing, listening, reading, and vocabulary building 

Each lesson consists of four sections: i) Writing - about 20 characters that learners 
can learn to draw with their fingers using the correct stroke order indicated on the 
screen, they can also hear how to pronounce them: ii) Listening test - learners hear 
ten randomly chosen words or phrases, then select the correct answer from the choices 
displayed; iii) Reading test - learners see ten randomly chosen words or phrases in char¬ 
acters, then select the correct answer from the choices displayed; and iv) word search 
- most lessons have two word search puzzles. Each puzzle consists of 16 characters that 
can form about eight words/phrases. (See Figure 1, below, for screen shots.) 

rlc Gaming feature - an element of fun 

Each listening and reading test is timed at 60 seconds as a default, which can be reset to 
40 or 20 seconds. The learner can see the clock count down, making it feel like playing a 
game. For word search puzzles, learners use their fingers to draw a line horizontally or 
vertically over the characters that form the correct word/phrase. If it matches the word/ 
phrase in the list, a pink line appears, otherwise, the line disappears. 


rtc Personalised learning 

Learners can set the level that suits them (easy, normal or hard). The default is Easy with 
stroke order indicator and 60 seconds per test. If users choose a harder level, the time 
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Figure 1. Screen shots for Writing, Listening and Reading sections 

and the Word search section 


available shortens and the stroke order indicator disappears. As users complete each 
character, data is kept within the device memory to allow the users to see how many 
times they have practised it. 

3.2 Development of the Chinese Characters First Steps app 

Although not much research had been done at the time on the potential of using mobile 
devices in language learning, the project team felt that the affordances of smartphones and 
24 tablets with touch screen were ideally suited to a language like Chinese. For the first time, 

































Rosell-Aguilar & Kan: A mobile application to teach Chinese characters 


it was possible to bring native speaker pronunciation, stroke-by-stroke character writing, 
pinyin, English definitions and instructions in one interactive mobile experience, instead 
of the traditional way of having a text book, dictionary, audio CD player and a notebook to 
write characters. It was felt that this integrated solution might better support and engage 
learners in maldng the four mappings discussed earlier in 2.1. 

The development of the app (iOS version) 1 went through three development phases: 

1. Phase I: Version 1 (February 2010 - September 2010): the app contained only the first 
20 characters in the course materials and it had three parts: stroke-by-stroke charac¬ 
ter writing, listening test and reading test. This version went live on the App Store in 
September 2010 as free download. 

2. Phase II: Version 2.0 - Version 2.1 (November 2010 - July 2011): a further 400 charac¬ 
ters were built with 20 lessons (lesson 1 free, lessons 2-20 after an upgrade via in-app 
purchase); and a word search' tool was added. This version went live on the App Store 
in July 2011. 

3. Phase III: Version 2.2 - Version 2.3 (June - August 2014): upgraded to meet iOS8 sys¬ 
tem reguirements and fix problems with upgrade button; the 'Listen' button added to 
each character in the Writing section; and the first five lessons free (lessons 6-20 after 
upgrade via in-app purchase). This version went live on the App Store in August 2014. 

The app has generated 30,766 downloads between January 2011 and May 2014. In that 
time, a total of 3,290 users have purchased the upgraded version with 20 lessons. In 
October 2014, the app won the Ed2.oWork competition - European Union-funded, edu¬ 
cation, Transversal KA3 ICT Network project (Lifelong Learning programme) Reference: 
519057-LLP-2011-UK-KA3-KA3NW. 

4. Research questions 

The research study reported here aimed to a) understand the learner experience of using 
the Chinese Characters First Steps app and b) evaluate the principles used for the design of 
the app. 

The researchers had gathered some first impressions from users of the app by looking 
at the reviews and ratings left on the UK and US App Stores. On the US App Store, 12 rat¬ 
ings were received, with an average of 3.5 stars (five 5-star, two 4-star, two 3-star, one 2-star 
and two l-star ratings). A further 12 ratings were posted on the UK store. There, the app 
received an average of 4 stars (seven 5-star, two 4-star and three l-star ratings). The positive 
reviews praised the appropriate level for beginners, variety of skills practised and the user 
interface. Low scores and negative reviews were received from users who had encountered 
technical problems with sound, tracing or payment (these reviews had been left prior to 
the bug fixes released with versions 2.2 and 2.3 of the app). Whilst this was of interest, a 
much more detailed account of the user experience was needed. 

Most studies into the use of apps for language learners have been carried out with the 
researchers’ own students as participants (e.g. Wong et al,20io; Yang and Xie,20i3). Teachers 
know their students and base their teaching strategy on that knowledge. When the design 
is aimed at massive online consumption, that knowledge is non-existent. Investment and 25 
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time is being spent on developing language learning apps, but developers, businesses and 
educators cannot design well without some knowledge of who users are and what they 
want. This research project aims to understand who the end-users of language learning 
apps are, using data both from students registered on the Oil Beginners' Chinese course 
and from other users external to the institution who had downloaded the app. 

The research questions were: 

1. What is the prohle of the app user? 

2. What is their motivation for using the app? 

3. How do they use the app? 

4. What are their expectations and impressions of using the app? 

5. Do they think that using the app has improved their Chinese language learning? 

5. Methods and participants 

5.1 Methods 

To obtain feedback from app users, a prompt was added to the app as a pop-up, which 
appeared after users utilised the app more than five times. In addition, an email was sent 
to a UK-based Chinese language teaching mailing list asking teachers to encourage their 
students to take the survey. Furthermore, a message with the survey link was posted on 
the course forum asking students registered on the OU beginners' Chinese course to take 
the survey, and finally a tweet with the survey link was sent from the course Twitter feed 
encouraging followers to take the survey. 

When users chose to take the survey, a link took them to the site where it was hosted. The 
survey (using SurveyMonkey) consisted of 25 questions, of which 24 were multiple-choice 
and one was an open comment question (see Appendix A for full survey). 

The survey ran between June 2012 and March 2014. A total of 137 responses were col¬ 
lected. All respondents finished the survey. The only question which did not receive full 
responses was the open question, which was answered by 69 participants. The data gath¬ 
ered were analysed using SPSS 21 to provide descriptive statistics. A second data analysis 
was carried out using a question which asked participants whether they were registered on 
a course in Chinese as a variable to ascertain whether there were any noticeable differences 
between formal and informal learners. Many of the responses in terms of use and impres¬ 
sions of quality were very similar. Where appropriate, the results section will highlight the 
responses where there were marked differences between the respondents. 

To further explore how learners utilised the app, four users of the app were inter¬ 
viewed in June 2013. The users were registered on the OU Beginners' Chinese course and 
had indicated that they had been using the app in response to another research study on 
mobile language learning (one of the authors of this paper participated in the study). The 
interviews were carried out using a semi-structured schedule (see Appendix B for sample 
questions) via Skype or telephone, with each interview lasting between 30 and 50 minutes. 
The interviews were recorded and transcribed. Using a thematic analysis of the interview 
transcriptions, the data were examined to identify salient features about the pedagogical 
design principles of the app and whether they facilitated the learning experience. 
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5.2 Participants 

The profile of the survey respondents is as follows: 79 (58.5%) were male and 56 (41.5%) 
female. With regards to age, the largest group of participants were aged between 30 and 39 
(Figure 2). The largest proportion of respondents live in the UK (34.3%), followed by other 
European countries (21.6%), North America (USA, Canada) (15.7%), China (11.2%), Australia 
and New Zealand (8.2%), other Asian countries (3.0%), Central and South America (2.2%), 
and Africa (0.7%). A further 3.0% indicated they live elsewhere. Only 17.2% of the respon¬ 
dents were registered on the OU beginners’ Chinese course, the remaining 82.8% were not. 

30% 



Under 25 25-29 30-39 40-49 50-59 60-64 65 and over 


Figure 2: Age of participants 


6. Results and discussion 

In this section the results from the data collected will be presented and discussed in rela¬ 
tion to the research guestions. 

6.1 What is the profile of the app user? 

A high proportion of the survey respondents were male (58.5%). This is in line with UK 
undergraduate population studying Chinese and consistent with the student profile of our 
university Chinese course (Kan & McCormick, 2014) as well as conventional UK universi¬ 
ties (HESA, 2011), where more men study Chinese in comparison with European languages. 

With regards to age, most participants were older than the traditional university-age stu¬ 
dents who had taken part in previously reported research, however, the age distribution 
percentages were very similar to those of the students taking the course at our institution, 
who are also older adult learners. 

Whereas most students taking the course live in the UK (as expected from a UK insti¬ 
tution), we found marked differences between those app users who were registered on a 
course and those who were not. As expected, most respondents who are registered on a 
formal course live in the UK because the OU is a UK-based university and therefore the app 27 
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is more widely promoted in the UK than throughout the rest of the world. Informal learners 
were much more highly represented in the US, Canada and China. This is not surprising as 
it would be unlikely for respondents who live in those countries to register with a UK uni¬ 
versity, whereas beginner Chinese learners who live elsewhere, China in particular, would 
be likely to want to support their learning with additional resources such as a mobile app. 

The majority of respondents (89.7%) described their current level of Chinese language as 
beginner (35.0% as “absolute beginner" and 54.7% as “beginner”) which is the audience the 
app is targeted at. A further 8.8% selected "intermediate" and two people selected a higher 
level, one (0.7%) “upper intermediate" and another one “advanced". 

6.2 What is their motivation For using the app? 

The survey asked participants why they were learning Chinese. The guestion allowed more 
than one answer and 69.3% responded that they were learning the language because of 
personal interest, as is common with many language learners (Coleman, 2009). In addition, 
46.7% did it because it was an intellectual challenge, and a further 33.6% indicated that it 
was relevant to their profession. Such a high proportion was expected given that China 
is a nation that has recently experienced enormous economic growth and now attracts a 
higher volume of business than previously. Heritage learners (9.5%) stated that they had 
family ties with China, and 8.8% were learning Chinese because it was relevant to their 
current studies. These are usually highly-motivated learners who actively seek additional 
resources such as mobile apps to support their learning. 

All respondents indicated that they used the app for learning rather than teaching. A 
total of 25.9% of respondents were formal learners registered on a Chinese course (either 
from the OU that the app supports or elsewhere), so the app was mostly used to learn 
Chinese informally: either as the main source of learning (as was the case for nearly 17% 
of the participants) or as support to other informal sources of learning (likely to be books 
or other online resources such as websites, podcasts and / or other apps). This conhrms 
Khaddage & Lattemann's finding (2013) that a vast majority of students who owned smart¬ 
phones use apps as additional learning resources. Among the users who were not registered 
on a Chinese course, the proportion of respondents who claimed that they were trying to 
learn informally was much higher (80.5%) than among those who were registered on a 
course (37.5%). Similarly, the difference between those who considered the app their main 
source of learning was much higher (24.4%) among those not registered than the respon¬ 
dents who were registered on a course (3.7%). 

As all the four interview participants were registered students of the OU Beginners’ 
Chinese course, they used the app as a revision tool to revise, reinforce and test the char¬ 
acters learned on the course. One participant (Interviewee A) used the app mostly for rec¬ 
ognising characters, saying it helped "to reinforce [...] to revise the characters”. Another 
participant (Interviewee C) used the app in conjunction with the course materials. He said 
that he tried "to learn all the words and do everything in line with the study" (see 6.3 for 
how they use the app). 

6.3 How do they use the app? 

Of the iOS devices that the app runs on, 70.8% used an iPhone, 32.1% an iPad, 11.7% an iPod 
28 touch and 5.1% selected “other”. Some used more than one device. 
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The survey asked the participants the number of lessons in the app they had completed 
at the time they took the survey. Just under three quarters of the respondents had used the 
app for more than one lesson, meaning they must have purchased the upgraded version 
with 20 lessons (including the 62% who knew they had upgraded and some of those who 
were unsure about whether they had upgraded or not). This is a high percentage, but must 
be taken with caution as it makes sense that users who had upgraded would make more 
use of the app and perhaps be more willing to take part in a survey about it. 

With regards to how long at a time users normally spend utilising the app, most respon¬ 
dents (72.8%) used the app for 15 minutes or less. Some 2.2% selected “less than 5 minutes", 
9.6% "about 5 minutes”, 36.0% “between 6 and 10 minutes”, 25.0% "around 15 minutes", 
17.6% "between 16 and 30 minutes" and 9.6% “over 30 minutes". This is consistent with the 
design principles for our app. 

The interview data shows that participants responded well to the bite-size approach to 
delivering content and the possibility of using the app during ‘dead time’. One interviewee 
(Interviewee A) commented that the app was "useful for filling in time”, whereas another 
(Interviewee B) mentioned she made sure she took her iPod with her during commute 
time so that she could use the app to learn characters. A further participant (Interviewee 
C) found that the advantage of having the app available was "the speed that you can get 
access" to fill in 10 or 15 minutes of spare time, which otherwise may be wasted. 

The interview data also shows that on the whole participants worked progressively 
lesson by lesson but at the same time took advantage of the app design to work on any 
section or character they wished to focus at a time. The design feature of being flexible 
and personalised within the framework of progressive learning was welcomed by all four 
participants. As Interviewee C commented that he “might just pick one [character], one of 
the ones I’ve done” sometimes, but overall he went lesson by lesson because “I want to keep 
doing it properly”. This is further supported by Interviewee B: “I think I just see what I fancy 
on the day and what I have been struggling with. If there is something that I know we have 
learnt but I can’t remember I will go back to that.” This further supports the findings in 
the literature on flexible design for distance language learning (Bates, 2005; Garrido, 2005). 
Garrido (2005) points out that in designing distance course 'there had to be a high degree 
of flexibility built into the course to meet wide-ranging expectations’ (2005,180); whilst at 
the same time distance learners need a teaching and learning framework that engenders 
a high level of motivation to help them stay on track’ (2005,184). 

6.4 What are their expectations and impressions of using the app? 

To elicit data on user impressions of the app, the survey asked participants about their 
expectations and how these were met. There was a wide range of expectations about the app. 
Based on the features of the app, four categories were offered as choices 1-4. In addition, a 
further two choices (5 and 6) were added involving an unrealistic expectation to become 
fluent in Chinese and having no expectations respectively. Respondents could tick all that 
applied. The results show that most respondents hoped to learn to write some characters, 
followed by learning to recognise some characters (see Table 2). 
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Table 2: User expectations 


"What expectations of the app did you have when you downloaded it?" 

Response % 

1 would learn to write some Chinese characters 

74.4% 

1 would learn to identify some Chinese sounds 

46.3% 

1 would learn to recognise some Chinese characters 

67.9% 

1 would learn to say a few Chinese words 

41.8% 

1 would be fluent in Chinese 

6.7% 

1 had no expectations 

9.7% 


It is interesting that a much lower percentage of respondents expected to learn to identify 
sounds (choice 2) or say words (choice 4). This could be linked to the app name and descrip¬ 
tion, both of which refer to characters rather than sounds or pronunciation, or there may 
be an assumption that using apps for learning Chinese is more apt for reading and writing 
rather than aural skills. It was interesting to see that a - very small - number of respon¬ 
dents indicated that they expected to become fluent in Chinese by using an app with 20 
lessons. On the one hand, this is clearly an unrealistic expectation - certainly in the case of 
this particular app - on the other hand, however, it suggests that some people may believe 
that studying a language with an app on their personal device can lead to fluency, which 
is something that would not have occurred five years ago. 

The next guestion asked participants whether the app had met their expectations (see 
Table 3). These results vary for those respondents who were registered on a Chinese course 
and those who were not. These differences between the two groups suggest that those not 
registered on a course had more realistic initial expectations of the app, as 53.8% found 
the app as they expected it to be. This seems logical for users who, as informal learners, 
are likely to utilise several apps and perhaps have more realistic expectations of what an 
app can do, which is consistent with the claim that learners often engage in meaningful 
language learning activities by using several apps (Lafford, 2011). Of the three respondents 
who selected “worse than expected", one had selected "I would be fluent in Chinese" as his/ 
her expectation. 

Table 3: Responses to the question "Has the app met your expectations?" 



All 

Registered 

Not registered 

Better than expected 

52.2% 

64.8% 

43.8% 

As expected 

45.5% 

33.3% 

53.8% 

Worse than expected 

2.2% 

1.9% 

2.5% 


It is pleasing that 97.7% of respondents indicated that the app had either met or exceeded 
their expectations, as it is evidence that the app is fit for purpose. Again, this supports 
findings from previous studies in the literature that users have positive attitudes towards 
using language learning apps (Kim, 2013; Khaddage & Lattemann, 20t3). 

In the open comment question, participants were asked what else they would like from 
the app. Some 69 respondents answered the question. The most popular request was 
30 for more characters/content, with 25 responses. This was predictable. Among these, six 
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specified they wanted content that was more challenging, including full sentences, a further 
six specified they wanted more content on tones, and three wanted more pronunciation 
practice. A total of four respondents reguested more free content. Some 16 respondents 
suggested new features: the most popular reguest (from six different respondents) was 
the ability to test themselves on writing characters without having a model answer in 
front of them. Other features suggested included the ability to interact with other learn¬ 
ers and an interface in a language other than English (two respondents each), a dictionary 
(two), being able to personalise the interface, and writing tests. Finally, four respondents 
would like detailed feedback on the guizzes. It is interesting that some of the suggestions 
for additional features, such as the ability to interact with others and detailed feedback, 
are consistent with features that had been highlighted in the literature as desirable from 
language learning apps. Some of the above suggestions have been addressed in the latest 
version of the app. 

All four participants in the interview said they liked the app. Interviewee C said “I really 
like that app [...] It's the favourite one that I’ve found". Depending on their own individual 
learning needs, they liked different functions of the app: some liked the stroke-by-stroke 
writing of characters most whilst others like the reading test/listening test or word search 
most. They also made some good suggestions for improvement such as adding audio but¬ 
ton to each character in the Writing section so that the user can listen to the character 
before they practise the writing, which was implemented into the latest version of the app. 

6.5 Do they think that using the app has improved their learning? 

Participants were asked to rate the overall guality of the app (see Table 4). With over 90% 
of participants rating the app as either 'good' or 'very good’, it was clear that they had very 
positive feelings about learning with it. 


Table 4: Responses to the question: "Please rate the overall quality of the Chinese 
Characters First Steps app" 



Overall 

quality 

Writing 

characters 

Reading 

characters 

Recognising 

words 

Very good 

51.1% 

42.5% 

54.1% 

50.4% 

Good 

39.3% 

29.9% 

33,8% 

35.6% 

OK 

6.7% 

23.1% 

11.3% 

11.9% 

Not so good 

3.0% 

4.5% 

0.8% 

2.2% 

Terrible 

0.0% 

0.0% 

0.0% 

0.0% 


Not all features were rated equally: whereas the quality of the app for learning to recognise 
Chinese characters and read Chinese words received very high ratings (over 86% rated it 
good' or very good’ for recognising characters and nearly 88% gave similar ratings for read¬ 
ing Chinese words - formed by several characters), the quality of learning to write Chinese 
characters received a generally positive but lower satisfaction rate (72.4% rated it ‘good’ 
or Very good’), possibly due to i) the frustration in drawing some complex characters on a 
small screen (as suggested by the interview data); and ii) the inherent difficulty of learning 
to write Chinese characters as opposed to just recognising them. 31 
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With respect to learning to write Chinese characters using the app, and whether this skill 
was transferable outside the app, participants were asked whether they thought that prac¬ 
tising writing characters on screen helped them to write on paper as well. In total, 48.9% 
selected "a lot", 31.9% “a little”, and 8.9% "not much". Nobody selected “not at all” and 10.4% 
had not tried to write the characters on paper after practising with the app. This suggests 
that participants believe that repeated writing of characters, whether using one’s Bngers 
on a screen or with pens on paper, facilitates the production of characters. 

Finally, the participants were asked whether using the app had improved their knowl¬ 
edge of Chinese. No gradation of improvement was offered as options other than “yes”, 
which 96.3% selected, or “no”, selected by 3.7%. The perceived benefits of learning characters 
with the app are further supported by the interview data, where all the four participants 
who had used the app found the app a useful learning and revision tool. Interviewee A 
reported that the app enabled him "to try and learn more of the characters more thor¬ 
oughly". For Interviewee B, she found the practice of stroke order useful “because 1 never 
remember which order the strokes go in. And also 1 like the reading practice because 
characters are difficult to remember." Interviewee D mentioned that using the app is "just 
a different way of reviewing characters and sounds”. As stated earlier, over 80% of partici¬ 
pants felt that practising writing characters on the app helped them, to some extent, to 
write characters on paper as well. The above findings are consistent with the literature that 
learners reported that learning with mobile devices helped the recognition and writing of 
Chinese characters (Huo, 2013; Chung 2013). 

7. Conclusion, limitations and further research 

7.7 Conclusion 

The results from this study have found that the app we created has been used in the way 
it was designed for and led to very positive impressions of learning with it. The app helps 
character recognition and writing as it combines the stroke-by-stroke writing of characters, 
sound, pinyin and English translation on one screen to reinforce the learning; it allows 
users to use the app when they have a small amount of time or when they are on the move; 
and by giving them the freedom as well as the structure to suit their own learning needs, 
they are more likely to engage with the learning. 

The study has also provided one of the first profiles of users of apps for language learn¬ 
ing and their usage habits. It is very positive that such a varied pool of users utilise the app, 
as it conhrms the appeal of apps for language learning regardless of gender or age. This 
research is necessary in the field of language learning apps to understand the way learners 
engage with them. The feedback and data collected from users for this study will be useful 
to inform the design of further developments. 

Sufficient evidence in the data suggests that the design principles used in the develop¬ 
ment of the app have been useful to learners. During the development of the app, one of 
the key challenges was to bridge the gap between the language teachers' limited techni¬ 
cal knowledge and the app developers’ limited of knowledge of both pedagogy and the 
Chinese language. It was crucial to communicate clearly and frequently on the pedagogy 
and learning design. The language experts needed to be ready to embrace new ideas but 
at the same time strike the right balance between technical affordances and pedagogy, for 
32 example by not spending an unnecessary amount of time trying to achieve some technical 
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novelty at the expense of the learning outcomes. Both sides had to be prepared to change 
the specification of the app as development went on because some ideas did not work out 
as expected, whilst other ideas came up during the development process. For example, the 
word search activity was not in the initial design of the app, but was borne during the 
process as we tried to combine the principle of creating an element of fun with progressive 
learning, which turned out to be a feature very much welcomed by users. Finally, whereas 
the academics were accustomed to taking longer periods of time to prepare resources, app 
developers worked at a fast pace. Because of this, it became important to respond guickly 
to gueries and suggestions and adapt to their availability to work on the app design. These 
are valuable lessons to future designers of language learning apps, regardless of whether 
they are commercial or educational content providers. 

7.2 Limitations 

The study asked participants to indicate what their impressions of learning with the app 
were. Whilst this is helpful to understand how users feel about using the app as a learning 
too, it does not provide any evidence of actual learning. In principle, the data from partici¬ 
pants who were taking the course could have been correlated with their performance on 
the course assessment. A number of obstacles impeded this. Firstly, some were using other 
sources of learning in addition to the app, so the issue of whether the learning was due to 
app use or not would have been unclear. Secondly, the confidentiality of users might be com¬ 
promised as it would be necessary to correlate the survey responses with their attainment. 
Thirdly, being distance learners, there would be no way of ascertaining that the person who 
took the survey was the same person taking the assessment. Finally, it was important to 
keep the survey confidential to avoid any perception of bias or participants thinking that 
taking part or "good" or "bad" responses may influence their grades. In addition, assess¬ 
ment scores would not have provided evidence of learning, as it is assumed that a large 
proportion of their learning is due to completing the course and not just based on the app. 

It was also impossible to gather analytics from the app itself on number of attempts or 
levels achieved and link it to the data from the survey. This is something that can be easily 
integrated into app design now but was not available at the time of the app development. 

7.3 Further research 

Work continues to improve the app: we have recently produced an Android version and 
updated the iOS version (released on the iTunes Store in August 2014) to meet new OS 
system reguirements and to include usage analytics. The data gathered from the analytics, 
user feedback and suggestions collected for this study will inform the design of further 
developments. We will continue to undertake research based on later versions of the app. 

We believe that further research is necessary in the field of apps for language learning, 
and apps for Chinese language learning in particular. More information is reguired to 
understand the way learners engage with apps for language learning, in particular when 
a proportion of these use apps as their only source of learning resources. Such research 
would benefit from screen recordings and eye-tracking technology as well as observation 
of behaviours and/or learning analytics of app activity (such as number of attempts or 
levels achieved, as discussed above). Although the participants in this survey reported very 
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positive impressions of learning, evidence of actual learning is needed. This could involve 

further research using control groups and pre-and post- learning activity tests. 

Notes 

i. Android version was developed after Phase II. This study is on iOS devices only, hence 
the Android development is not included 
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Appendix A 

Online survey questions 

Chinese Characters First Steps app survey. This survey aims to find out more about the 
people who download The Open University “Chinese Characters First Steps" App and how 
they use it. Your answers will be completely anonymous. You will not be identifiable and we 
will not ask you for any contact information. It should take less than 5 minutes to complete. 

By completing this survey, you give permission for the data collected to be used in an 
anonymous form in any written reports, presentations and published papers relating to 
this study for research purposes. 

1. Why are you learning Chinese? (tick all that apply) 

Personal interest 

Intellectual challenge 
I have family ties with China 
Relevant to my profession 
Relevant to my current studies 

2. How would you describe your overall level in Chinese at the moment? 

Absolute beginner 

Beginner 

36 Intermediate 
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Upper intermediate 
Advanced 
Near native 
Native 

3. Are you currently registered or have you ever taken the L197 Beginners' Chinese course 
from The Open University? 

Yes 

No 

4. Are you currently registered on a Chinese course (not at the OU)? 

Yes (go to guestion 6) 

No 

5. Are you trying to learn Chinese “informally" (i.e. by yourself without signing up to a 
course)? 

Yes 

No 

6. On which of the following devices do you use the Chinese Characters First Steps App? 
(tick all that apply) 

An iPod touch 
An iPhone 
An iPad 
Other 

7. Do you use the Chinese Characters First Steps App for: 

Learning (1 want to learn Chinese) 

Teaching (1 teach students or family members) [go to guestion 15] 

8. Do you regard the Chinese Characters First Steps App as: 

Additional to other learning 1 do 

Main source of learning 

9. Have you upgraded your app to unlock all the content? 

Yes (go to guestion 11) 

No 

Don’t know 

10. Why haven't you upgraded your app to unlock all the content? 

Too expensive 

Not worth it 

1 want to know more about the content before 1 pay for it 
1 never pay for apps 

1 wasn’t sure how much I'd have to pay if I tried to upgrade 
1 tried but had technical problems doing so 
1 was not aware that I could upgrade the App 
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u. How many of the lessons in the App have you attempted? 

1 

2-5 

6-io 

11-15 

16-20 

12. How many of the lessons in the App have you completed? 

l 

2-5 

6-io 

11-15 

16-20 

13. How long do you normally spend using the App at a time? 

Less than 5 minutes 

About 5 minutes 
Between 6 and 10 minutes 
Around 15 minutes 
Between 16 and 30 minutes 
Over 30 minutes 

14. What expectations of the app did you have when you downloaded it? (tick all that apply) 
1 would learn to write some Chinese characters 

1 would learn to identify some Chinese sounds 
1 would learn to recognise some Chinese characters 
1 would learn to say a few Chinese words 
1 would be fluent in Chinese 
1 had no expectations 

15. Has the app met your expectations? 

Better than expected 

As expected 
Worse than expected 

16. Please rate the overall quality of the Chinese Characters First Steps App 
Very qood 

Good 

OK 

Not so good 
Terrible 

17. Do you think that practising writing the characers on screen helps to learn to write 
them on paper as well? 

A lot 
A little 
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Not at all 
1 haven't tried 

18. Please rate the quality of the Chinese Characters First Steps app for learning to write 
Chinese characters 

Very good 

Good 

OK 

Not so good 
Terrible 

19. Please rate the quality of the Chinese Characters First Steps app for learning to read 
Chinese characters 

Very good 

Good 

OK 

Not so good 
Terrible 

20. Please rate the quality of the Chinese Characters First Steps app for learning to recognise 
Chinese words 

Very good 

Good 

OK 

Not so good 
Terrible 

21. Has using the Chinese Characters First Steps app improved your knowledge of Chinese? 
Yes 

No 

22. What else would you like to see available in the app? 

23. How old are you? 

Under 25 
25-29 

30-39 
40-49 
50-59 
60-64 
Over 65 

24. Areyou... ? 

Male 

Female 
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25.Where do you live? 

UK 

Other European Union country 
Other European country (non-EU) 

North America (US, Canada) 

Central / South America 

Africa 

Middle East 

China 

Japan 

Another Asian country 
Australia/New Zealand 
Other 

Your participation will help us to improve our understanding of app design and what users 
want. Many thanks for taking part. 

Appendix B 

Sample semi-structured interview questions 

General questions about using mobile devices for language learning 

What land of mobile devices do you use for language learning? 

Do you switch between devices? What kind of language activities do you use the iPhone 
for, for example? 

Because of the mobile devices you have now, do you find yourself spending more or less 
time on language learning? 

Questions about the 'Chinese Character First Steps' app 
Why do you use this app? 

Which section of the app do you find most useful (i.e. Writing, Reading Test, Listening 
Test and Word search)? 

Where and when do you use the app? 

There are 20 lessons on the app. Do you use them in the order presented or randomly? 

Blue sky thinking - any features you would like to see in the app which are currently not 
there? 
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